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POINTING DEVICE HAVING FINGERPRINT IMAGE RECOGNITION 
FUNCTION, FINGERPRINT IMAGE RECOGNITION AND POINTING 
METHOD, AND METHOD FOR PROVIDING PORTABLE TERMINAL 
SERVICE USING THEREOF 
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TECHNICAL FIELD 

The present invention generally relates to a pointing device having a fingerprint 
image recognition function and a fingerprint recognition method thereof, and more 
specifically, to a pointing device for performing a user recognition and a point control by 
10 using one kind of sensors without using both a user recognition sensor and a point control 
sensor and a fingerprint image recognition method thereof. 

BACKGROUND ART 

In general, a pointer that is a pointing device refers to a XY tablet, a trackball 

15 and a mouse which have been widely used in a desktop computer or to a touch screen 
panel or a touch pad which have been widely used in a portable terminal device such as a 
laptop computer. Recently, an optical mouse using light has been used. 

Attempts to integrate biometrics into electronics and communications equipment 
and its peripheral equipment have recently increased. The most remarkable 

2 0 characteristic of the biometrics is to relieve troubles of loss, stealing, oblivion or 
reproduction resulting from external factors in any case, and this characteristic is most 
advantageous. When the characteristic is used, an audit function is completely 
performed to trace who violates security. Specifically, a user recognition technology 
with a fingerprint have been actively commercialized, and it is easy to access and carry 

2 5 the user recognition technology because it recognizes a user by using a characteristic of 
human. As a result, various studies have been made and also much development has 
been made in this field. 

Recently, a technology where user recognition using a fingerprint is introduced 
to a pointing device has been developed. In the pointing device, an inner fingerprint 

30 recognition device recognizes a fingerprint from a finger surface through a predetermined 
window, and compares a previously registered fingerprint with the recognized fingerprint 
to certify the fingerprint when the comparison result is identical independently of a 
pointing function. Fig. 1 shows a fingerprint recognition optical mouse for an example. 
As shown in Fig. 1, the fingerprint recognition optical mouse 1 has the same shape and 

35 the same function as those of a general mouse, but comprises a fingerprint recognition 



window 2 in a portion whereon a right thumb touches. If the right thumb touches the 
fingerprint recognition window 2, the inner fingerprint recognition sensor (not shown) 
recognizes a fingerprint of the thumb and compares a previously registered fingerprint 
with the recognized fingerprint to determine recognition of a user. 

In case of a conventional fingerprint recognition, a fingerprint having the least 
size necessary in user recognition is to be acquired. That is, in case of the fingerprint 
recognition optical mouse of Fig. 1, a fingerprint of about 100 x 100 pixels for fingerprint 
recognition is to be acquired. Currently, an optical mouse for detecting a fingerprint 
image of 96 x 96 pixels at one time has been commercialized in the market. 

Also, the fingerprint recognition may be introduced to a pointing device using a 
fingerprint. In other words, a technology of controlling a pointer using a fingerprint and 
recognizing a user with the fingerprint at the same time has been provided in the current 
pointing device. However, in the prior art, both a fingerprint acquiring sensor for 
fingerprint recognition to acquire a larger fingerprint image for user recognition and a 
fingerprint acquiring sensor for controlling a pointer to acquire a smaller fingerprint 
image are comprised in the one pointing device. 

Fig. 2 shows a portable terminal device comprising two fingerprint acquiring 
sensors. Fig. 2(a) illustrates a part of a portable computer (laptop computer) and Fig. 
2(b) illustrates a part of a PDA. In case of Fig. 2(a), a fingerprint acquiring sensor 3 for 
fingerprint recognition which recognizes a fingerprint of a user to certify the fingerprint 
of the user and a pointer controlling sensor 4 for controlling a pointer represented in a 
monitor of a laptop computer with a finger are comprised. However, a pointer control in 
the laptop computer is not to use fingerprint recognition but to use change of capacitance 
by pressure of a fmger or a stylus. Also, in case of Fig. 2(b), a fingerprint acquiring 
sensor 5 for user recognition and a pointer controlling sensor 6 are comprised, 
respectively. Generally, in case of a laptop computer for controlling movement of a 
cursor on a monitor depending on movement information of a finger, the fingerprint 
image of about 20 x 20 pixels is sufficient to obtain the movement information of the 
fingerprint for point control, but for user identification, the fingerprint image of more than 
about 100 x 100 pixels is required. In case of the actual laptop computer, a product to 
acquire image data of about 100 x 100 pixels with a fingerprint acquiring sensor of about 
5 x 5mm has been commercialized. 

Although a low power high speed is embodied by using small data of about 20 x 
20 pixels for point control, it is necessary to acquire and analyze a large fingerprint image 
of 100 x 100 pixels for fingerprint recognition. As a result, fingerprint acquiring sensors 



3 and 5 for user recognition and pointer controlling sensors 4 and 6 are required. Since 
two kinds of fingerprint acquiring sensors are comprised in the pointing device, the 
exterior of the pointing device does not look good and the technical complexity for 
driving the two fingerprint acquiring sensors cannot be solved. Therefore, since two 
kind of fingerprint acquiring sensors are mounted in the prior art, adverse effects are 
caused on part miniaturization as electronic devices and apparatuses become thinner and 
simpler. 

Accordingly, a method for performing a user recognition by using a fingerprint 
acquiring sensor for user recognition in a portable terminal device and performing a point 
control by acquiring a plurality of small fingerprint images at the same time has been 
required. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a perspective view of a conventional optical mouse for fingerprint 
recognition. 

Fig. 2 is a diagram illustrating an example of a portable terminal device 
comprising a conventional fingerprint sensor for fingerprint recognition and a 
conventional navigation pad for pointer control. 

Fig. 3 is a diagram illustrating a structure of a pointing device according to a first 
embodiment of the present invention. 

Fig. 4 is a diagram illustrating a process of calculating displacement data 
according to the present invention. 

Fig. 5 is a diagram illustrating a process of mapping a fingerprint image 
according to the present invention. 

Fig. 6 is a diagram illustrating a structure of a pointing device according to a 
second embodiment of the present invention. 

Fig. 7 is a diagram illustrating a process of mapping fingerprint images acquired 
from a plurality of fingerprint acquiring means shown in Fig. 6. 

Fig. 8 is a flow chart illustrating a fingerprint recognition process in a pointing 
device according to the first or the second embodiment of the present invention. 

Fig. 9 is a detailed flow chart illustrating a process of mapping a fingerprint 
image in a virtual image space in Fig. 8. 

Fig. 10 is a flow chart illustrating a process of controlling a pointer in a pointing 
device according to the present invention. 

Fig. 1 1 is a diagram illustrating a structure of a pointing device according to a 



third embodiment of the present invention. 

Fig. 12 is a diagram illustrating a design example of 3 : I reduction optics 
mountable in a microscopic space according to the present invention. 

Fig. 13 is a diagram illustrating an example of a fingerprint image acquired from 
5 a fingerprint acquiring means by applying the reduction optic system thereto according to 
the present invention. 

Fig. 14 is a diagram illustrating a pointing device according to a fourth 
embodiment of the present invention. 

Fig. 15 is a diagram illustrating a method for extracting a fingerprint image of m 
10 x n pixels from that ofMxN pixels. 

Fig. 16 is a flow chart illustrating a method for performing a user recognition and 
a pointing control at the same time according to the third or the fourth embodiment of the 
present invention. 

Fig. 17 is a diagram illustrating a structure of a pointing device according to a 
1 5 fifth embodiment of the present invention. 

Fig. 18 is a flow chart illustrating the operation of the pointing device according 
to the fifth embodiment of the present invention. 

Fig. 19 is a flow chart illustrating a method for limiting usage of a portable 
communication terminal device depending on users by using a fingerprint recognition 
2 0 function according to the present invention. 

Fig. 20 is a diagram illustrating an example of a portable terminal device 
comprising a pointing device according to the present invention. 

DETAILED DESCRIPTION OF THE INVENTION 

2 5 Technical Subject 

It is an object of the present invention to improve a fingerprint recognition 
method, thereby simultaneously performing a user recognition and a pointer control with 
only one kind of sensors without requiring respective fingerprint recognition sensors for 
the user recognition and for the pointer control. 
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Technical Solution 

In an embodiment, a pointing device having a fingerprint image recognition 
function comprises: at least one or more fingerprint acquiring means for acquiring a 
fingerprint image of a finger surface depending on a predetermined cycle; a characteristic 

3 5 point extracting means for extracting at least one or more characteristic points from the 
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acquired fingerprint image; a movement detecting means for calculating displacement 
data between characteristic points of the extracted fingerprint image to detect movement 
information of the fingerprint image; a mapping means for mapping the fingerprint image 
in an inner virtual image space depending on the movement information; a recognizing 
means for comparing a previously registered fingerprint image with the whole mapped 
fingerprint image when the entire size of the mapped fingerprint image reaches a 
previously set size and determining recognition on the fingerprint; and an operating 
means for receiving the displacement data from the movement detecting means and 
calculating a direction and a distance where a pointer is to move with the displacement 
data. 

In an embodiment, a pointing device having a fingerprint recognition function 
comprises: a fingerprint acquiring means (first operating cycle) for acquiring a fingerprint 
image of a finger surface which controls a pointer through only once 2-dimensional image 
acquisition; a fingerprint recognizing unit (second operating cycle) for comparing 
characteristic points of the previously registered fingerprint image with those of the 
acquired fingerprint image to recognize a user of the acquired fingerprint image; and a 
pointing control unit (third operating cycle) for detecting movement information based on 
partial data of the image acquired depending on the first operating cycle and calculating 
displacement data of the fingerprint image depending on the movement information to 
calculate movement direction and distance of the pointer. 

In an embodiment, a method for recognizing a fingerprint for user recognition 
comprises the steps of: acquiring at least one or more fingerprint images with a 
predetermined fingerprint acquiring sensor depending on a set cycle; extracting at least 
one or more characteristic points from the acquired fingerprint image; mapping a first 
fingerprint image in a specific location of a virtual image space; calculating displacement 
data between characteristic points of the first fingerprint image and those of a second 
fingerprint image acquired in the next cycle after the cycle where the first fingerprint 
image is acquired; mapping the second fingerprint image with the displacement data in 
the virtual image space; and comparing characteristic points of the previously registered 
fingerprint image with those of the whole mapped fingerprint image when the whole size 
of the fingerprint images mapped in the virtual image space reaches a previously set size, 

and determines recognition of the fingerprint. - 

In an embodiment, a pointing method of a pointer control device with an image 
sensor having a smaller size than a predetermined size required in a fingerprint 
recognition comprises the steps of: acquiring at least one or more fingerprint images of M 



x N pixels depending on a first operating cycle with a predetermined fingerprint acquiring 
sensor on a finger surface which controls a movable pointer; determining recognition of a 
user of the fingerprint image by extracting characteristic points from the acquired 
fingerprint image depending on a second operating cycle and comparing the extracted 
characteristic points with those of the previously register fingerprint image; extracting a 
fingerprint image of m x n pixels from the acquired fingerprint image depending on a 
third operating cycle; detecting movement information of the respective fingerprint image 
by calculating displacement data of the extracted fingerprint image ofmxn pixels; and 
calculating and outputting a direction and a distance where the pointer is to move with the 
displacement data. 

In an embodiment, a pointing device having a fingerprint recognition function 
comprises: at least one or more fingerprint acquiring means for acquiring an image of a 
finger surface depending on a predetermined cycle or on occasional requirement; a 
movement detecting means for calculating displacement data from the acquired 
fingerprint image to detect movement information of each fingerprint image; an operating 
means for receiving the displacement data from the movement detecting means to, 
calculate direction and distance where a pointer is to move using the displacement data; a 
storing means for mapping fingerprint images obtained from the fingerprint acquiring 
means and the operating means, and the displacement data of the fingerprint images; a 
CPU for analyzing and processing data of the operating means and an image storing space. 

Preferred embodiments 

The present invention will be described in detail with reference to the 
accompanying drawings. 

Fig. 3 is a diagram illustrating a structure of a pointing device according to a first 
embodiment of the present invention. 

The pointing device of Fig. 3 comprises an light emitting means 22, a light 
gathering means 23, a fingerprint acquiring means 24, a characteristic point extracting 
means 25, a memory means 26, a movement detecting means 27, a mapping means 28, a 
virtual image space 29, a recognizing means 30 and an operating means 31. 

Referring to Fig. 3, when a finger 20 to acquire a fingerprint image touches a 

transparent member 21, the light emitting means 22 emits light with the surface -of- the 

finger 20 which is a touch object. The light emitting means 22 includes at least one or 
more light emitting diodes. 

The light gathering means 23 condenses a fingerprint image generated by light 



emitted to the touch object from the light emitting means 22. An optical convex can be 
used as the light gathering means 23. 

The fingerprint acquiring means 24 detects an analog fingerprint image 
condensed by the light gathering means 23 and converts the analog fingerprint image into 
a digital fingerprint image. The fingerprint acquiring means 24 includes an optical 
sensor array where a plurality of CMOS image sensors (abbreviated as "CIS") are 2- 
dimensionally arranged. Here, the fingerprint acquiring means 24 acquires a plurality of 
fingerprint images at a previously set cycle. The fingerprint acquiring means 24 is 
manufactured to be suitable for a mini-portable terminal device, thereby acquiring a small 
fingerprint image. For example, a micro sensor to acquire a fingerprint image of less 
than about 20 x 20 pixels suitable for pointing control is used as the fingerprint acquiring 
means 24. Here, devices known to a person having an ordinary skill in the art can be 
used for the light emitting means 22, the transparent member 21, the light gathering 
means 23 and the fingerprint acquiring means 24. 

Light emitted from the light emitting means 22 is mirrored to the surface of the 
finger 20, and reflected depending on patterns of the surface of the finger 20. The light 
reflected from the bottom surface of the finger 20 forms a phase in the fingerprint 
acquiring means 24 through the light gathering means 23. The phase formed in the 
fingerprint acquiring means 24 is converted into a digital fingerprint image by the 
fingerprint acquiring means 24. The acquisition of fingerprint images is continuously 
performed at a rapid speed on a time axis. 

The characteristic point extracting means 25 extracts at least one or more 
characteristic points from each fingerprint image acquired from the fingerprint acquiring 
means 24 in a predetermined cycle. These characteristic points includes ridge length 
and direction of fingerprint images, location data where ridges are separated or ended. 

The memory means 26 stores fingerprint images acquired from the fingerprint 
acquiring means 24 and information on characteristic points extracted from the 
characteristic point extracting means 25. 

The movement detecting means 27 detects the degree of movement of each 
fingerprint image from characteristic points of fingerprint images stored in the memory 
means 26. Here, the movement detecting means 27 detects the degree of movement of 
fingerprints by calculating displacement data (direction and distance) of characteristic 
points depending on movement of fingerprints with a motion estimation method. In this 
way, the movement detecting means 27 detects the degree of movement of fingerprint 
images by comparing characteristic points of the fingerprint images acquired in a 



previously set cycle. The movement information of fingerprint images and the 
characteristic point extraction in fingerprint recognition as well as the acquisition of 
fingerprint mages are importance factors because the movement of fingerprint images and 
the reliability of the fingerprint recognition are differentiated depending on how reliably 
characteristic points can be extracted. 

The mapping means 28 receives displacement data (direction and distance) of 
characteristic points of fingerprint image, which are the movement information depending 
on movement of the fingerprint image from the movement detecting means 27, and 
determines a location where the moved fingerprint image is to be mapped in the virtual 
image space 29 with the displacement data. Next, the mapping means 28 maps each 
fingerprint image depending on the determined location. When the fingerprint image is 
mapped by the mapping means 28, the same characteristic points are preferably mapped 
to be superposed among the characteristic points acquired at the previous cycle and at the 
current cycle. In this way, the mapping means 28 2-dimensionally arranges fingerprint 
images acquired at every time in the virtual image space. Here, the virtual image space 
29 has the size of the fingerprint image required in user recognition. That is, the virtual 
image space 29, which is a memory device for synthesizing fingerprint images required in 
user recognition, preferably has the size of the fingerprint image required in user 
recognition. For example, the virtual image space 29 has the size of less than about 100 
x 100 pixels. 

The recognizing means 30 detects whether the size of the whole fingerprint 
image mapped in the virtual image space 29 is identical with that of the virtual image 
space 29, and then compares the previously registered fingerprint image with the whole 
mapped fingerprint image if the size is identical to certify a user. 

The operating means 31 receives displacement data from the movement 
detecting means 27, and calculates a direction, a distance and a movement degree where 
the pointer is to move with the displacement data. The operating means 30 is generally 
combined with a pointing device or with a processor of apparatus having the pointing 
device. As a result, the processor can control the pointer to move in a desired direction 
and at a desired distance on a screen of a display device. . 

In an embodiment, the fingerprint acquiring means 24 can be embodied in 
various ways. That is, the fingerprint acquiring means 24 can be embodied with" a' ' 
semiconductor device or with an optical system as described above. Here, the 
fingerprint acquiring means 24 using the optical system has been commercialized through 
a verification system for a long period, and is advantageous in scratch, temperature and 
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durability. However, the optical system has a limitation in its usage to a mini-portable 
terminal device due to the size of an optical sensor, and has a problem of impossibility of 
information security and recognition adoption. Meanwhile, the fingerprint acquiring 
means 24 using a semiconductor device has a clear picture image and a rapid response 
speed when fingerprint image are acquired. Also, since miniaturization of the sensor is 
possible, the fingerprint acquiring means 24 using a semiconductor device has various 
application fields and large competitiveness in cost. 

In an embodiment, the acquisition of fingerprint images can be performed with 
the optical system or the semiconductor system. For example, when fingerprint images 
are acquired with the semiconductor system, the above-described light emitting means 22 
and light gathering means 23 are not required. As a result, Fig. 3 shows an example of 
the pointing device for acquiring fingerprint images with the optical system, and also 
fingerprint images can be acquired with the semiconductor system. Since the present 
invention is characterized not in acquisition of fingerprint images but in a processing 
method of the acquired fingerprint images, the fingerprint acquiring method can be 
performed with any system. 

Hereinafter, the operating of the pointing device having a fingerprint recognition 
function will be described in detail. In an embodiment of the present invention, the 
fingerprint recognition function and then the pointing function are described because the 
fingerprint recognition function and the pointing function are simultaneously performed 
in the pointing device. 

Fig. 4 is a diagram illustrating a process of calculating displacement data 
according to the present invention. Fig. 4a shows a fingerprint image and its 
characteristic points acquired in a first cycle, and Fig. 4b shows a fingerprint image and 
its characteristic points acquired in a second cycle. The fingerprint images of Figs. 4a 
and 4b are images formed in the fingerprint acquiring means 24. In an embodiment, the 
fingerprint image where 5 characteristic points (represented as M) are extracted is 
illustrated for an example. 

The fingerprint image of Fig. 4b is obtained by moving the fingerprint image of 
Fig. 4a rightward 3 pixels (AX=3) and downward 3 pixels (AY— 3) for a predetermined 
cycle. The movement detecting means 27 grasps the movement of the fingerprint image 
by calculating displacement data (direction and distance) of the extracted characteristic 
points. 

The mapping means 28 maps fingerprint images acquired at every time in a 
corresponding location of the virtual image space 29 with the displacement data of the 
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characteristic points calculated by the movement detecting means 27. The mapping 
process is described in detail with reference to Fig. 5. 

Fig. 5 is a diagram illustrating a process of mapping a fingerprint image 
according to the present invention. 
5 In an embodiment, the process of acquiring a fingerprint image having a size (i.e., 

less than about 100 x 100 pixels) required in user recognition with the microminiaturized 
fingerprinting acquiring means 24 for acquiring a fingerprint image of less than 20 x 20 
pixels is described. 

Fig. 5a shows fingerprint images acquired depending on the previously set cycle 

10 with the microminiaturized fingerprint acquiring means 24 of less than 20 x 20 pixels, 
and Fig. 5b shows the virtual image space 29 of less than 100 x 100 pixels where the 
fingerprint images are mapped. In Fig. 5, suppose that predetermined figures are not 
shapes of fingerprints but patterns of figures for convenience of explanation. 

When the fingerprint acquiring means 24 acquires a first fingerprint image 41 of 

15 20 x 20 pixels at a timing T 0 , the characteristic pint extracting means 25 extracts at least 
one or more characteristic points from the acquired first fingerprint image 41 and stores 
the characteristic points in the memory means 26. In Fig. 5a, 6 characteristic points 
(represented as black spots) are extracted on the first fingerprint image 41. The mapping 
means 28 maps the first fingerprint image 41 at a predetermined location of the virtual 

2 0 image space 29. Here, the mapping means 28 preferably maps the acquired fingerprint 
images at the center of the virtual image space 29. Thereafter, when the fingerprint 
acquiring means 24 acquires a second fingerprint image 42 at a timing T l5 the 
characteristic point extracting means 25 extracts at least one or more characteristic points 
(number: 8) from the second fingerprint image 42 and stores the characteristic points in 

2 5 the memory means 26. 

The movement detecting means 27 calculates displacement data (direction and 
distance) with the movement information of the first fingerprint image 41 and the second 
fingerprint image 42. The displacement data are calculated by the method described in 
Fig. 4. The mapping means 28 maps the second fingerprint image 42 in locations of the 

30 virtual image space 29 corresponding to the calculated displacement data in the 
movement detecting means 27. Then, when the fingerprint acquiring means 24 acquires 
a third fingerprint image 43 at a timing T 2 , the characteristic point extracting means "25' " 
extracts at least one or more characteristic points (number: 9) from the third fingerprint 
image 43. These extracted characteristic points are stored in the memory means 26, and 

35 displacement data on the second fingerprint image 42 and the third fingerprint image 43 



are calculated with the extracted characteristic points. The mapping means 28 maps the 
third fingerprint image 43 in locations of the virtual image space 29 corresponding to the 
calculated displacement data. As described above, the fingerprint image acquisition, the 
characteristic point extraction, the displacement data calculation and the mapping 
operation are repeatedly performed depending on a predetermined cycle until the whole 
size of the mapped fingerprint images 41, 42, 43, n is a size required in user 
recognition, that is, that of the virtual image space 29. In this way, the fingerprint image 
having a large size required in user recognition can be obtained with a plurality of 
fingerprint images each having a small size. 

During the mapping operation, when the fingerprint image acquired in the 
current cycle is mapped in the virtual image space 29, it is preferable to map 
characteristic points so that at least a part of the characteristic points of the fingerprint 
image acquired in the previous cycle may be superposed with that of the fingerprint 
image acquired in the current cycle. For example, when the second fingerprint image 42 
is mapped in the virtual image space 29 in Fig. 5b, at least parts of the characteristic 
points of the second fingerprint image 42 are mapped to be superposed with that of the 
characteristic points of the first fingerprint image 41. In the same way, when the third 
fingerprint image 43 is mapped in the virtual image space 29, at least a part of the 
characteristic points of the third fingerprint image 43 are mapped to be superposed with 
that of the characteristic points of the second fingerprint image 42. In Fig. 5b, the 
reference number 48 represents a portion where the first fingerprint image 41 is 
superposed with the second fingerprint image 42, and the reference number 49 represents 
a portion where the second fingerprint image 42 is superposed with the third fingerprint 
image 43. 

Meanwhile, the second fingerprint image 42 is obtained by moving the finger 20 
for a predetermined time (T1-T0) after acquisition of the first fingerprint image 41. In 
Fig. 3, when the finger 20 moves in a random direction at a random distance while 
touching the transparent member 21, the fingerprint image is acquired depending on the 
set cycle. In this case, the movement direction of the finger 20 is opposite to that of the 
fingerprint image. 

As described above, the fingerprint images acquired at every time in the set cycle 
are mapped in the virtual image space 29. When the whole size of the mapped " 
fingerprint images is identical with that of the virtual image space 29, the recognizing 
means 30 compares the previously registered fingerprint image with the whole fingerprint 
image mapped in the virtual image space 29 and then determines identification. Here, 



the identification is preferably determined through characteristic point matching of the 
fingerprint images. The recognizing means 30 certifies a user if the two fingerprint 
images are identical but refuses user certification if not. As a result, the user can restrict 
usage of the terminal device so that only the user can use or prevent information that the 
user intends to protect from leaking in a device. 

Fig. 6 is a diagram illustrating a structure of a pointing device according to a 
second embodiment of the present invention. 

The pointing device of Fig. 6 comprises a plurality of light emitting means and 
fingerprint acquiring means more than that of Fig. 3. In Fig. 6, the plurality of 
fingerprint acquiring means 24-1, 2, 3 acquire a plurality of fingerprint images depending 
on a predetermined cycle at every time. In comparison with Fig. 3, while the fingerprint 
acquiring means 24 acquires one fingerprint image at every cycle in the pointing device 
of Fig. 3, each of the plurality of fingerprint acquiring means 24-1, 2, 3 acquires a 
fingerprint image so that a plurality of fingerprint image are acquired at every cycle in the 
pointing device of Fig. 6. Fig. 6 illustrates the pointing device comprising 3 light 
emitting means 23-1, 2, 3 and 3 fingerprint acquiring means 24-1, 2, 3. 

Fig. 7 is a diagram illustrating a process of mapping fingerprint images acquired 
from the plurality of fingerprint acquiring means 24-1, 2, 3 shown in Fig. 6. 

Fig. 7 shows the process of acquiring a fingerprint image of about 20 x 20 pixels 
with 3 microminiaturized fingerprint acquiring means and then acquiring a fingerprint 
image having a size (about 100 x 100 pixels) required in user recognition using the 
fingerprint image of about 20 x 20 pixels. Fig. 7a shows the fingerprint image of 20 x 
20 pixels acquired by the fingerprint acquiring means 24-1, 2, 3 depending on the 
previously set cycle. Fig. 7b shows the process of mapping the fingerprint images 
shown in Fig. 7a corresponding to locations of the virtual image space 29 of 100 x 100 
pixels. 

When the mapping process of Fig. 7 is compared with that of Fig. 5, the mapping 
process of Fig. 5 maps one fingerprint image acquired at every time in the virtual image 
space 29 depending on the set cycle while that of Fig. 7 maps 3 fingerprint images 
acquired at every time in the virtual image space 29 depending on a predetermined cycle. 

Referring to Figs. 7a and 7b, the fingerprint acquiring means 24-1, 2, 3 
simultaneously acquire 3 fingerprint images of 20 x 20 pixels (first fingerprint image' set 
61) at a timing T 0 . Next, the characteristic point extracting means 25 extracts at least 
one or more characteristic points from the first fingerprint image set 61, and stores the 
extracted characteristic points in the memory means 26. The characteristic points of the 
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first fingerprint image set 61 in Fig. 7a are all 12 (represented as black spots). The 
mapping means 28 maps the first fingerprint image set 61 in a specific location of the 
virtual image space 29. Here, the first fingerprint image set 61 is preferably mapped at 
the center of the virtual image space 29. Thereafter, the fingerprint acquiring means 24- 
5 1, 2, 3 acquire 3 fingerprint images (second fingerprint image set 62) at the next timing 
Tl. The characteristic point extracting means 25 extracts at least one or more 
characteristic points (number: 9) from the second fingerprint image set 62, and stores the 
characteristic points in the memory means 26. The movement detecting means 27 
calculates displacement data (direction and distance) with movement information of the 

10 first fingerprint image set 61 and the second fingerprint image set 62. The displacement 
data are calculated by the same method described in Fig. 4. The mapping means 28 
maps the second fingerprint image set 62 in locations corresponding to the displacement 
data calculated by the movement detecting means 27. The mapping operation is 
repeatedly performed until the whole size of the mapped fingerprint image sets 61, 62, . 

15 n becomes a size required in user recognition, that is, the size of the virtual image space 
29. As a result, the fingerprint image having a large size required in the user recognition 
can be obtained with a plurality of fingerprint images each having a small size. Here, 
the second fingerprint image set 62 includes fingerprint images obtained from the 3 
fingerprint acquiring means 24-1, 2, 3 by moving a finger for a predetermined time Ti-T 0 

2 0 after acquisition of the first fingerprint image set 6 1 . 

Here, as described in Fig. 5, when the fingerprint image sets acquired at the 
current cycle are mapped in the virtual image space 29, it is preferable to map 
characteristic points so that a part of characteristic points of the fingerprint image set 
acquired at the current cycle may be superposed with that of characteristic points of the 

2 5 fingerprint image acquired at the previous cycle. 

The recognizing means 30 determined identification by comparing the 
previously registered fingerprint image with the whole fingerprint image mapped in the 
virtual image space 29 when the fingerprint image sets are mapped in the entire virtual 
image space 29. The recognizing means 30 certifies a user when the two fingerprint 

30 images are identical, but refuses the user certification if not. 

Meanwhile, the pointing device according to an embodiment of the present 
invention controls a pointer with movement of fingerprint images acquired from" the' : 
finger surface. The pointing process in the pointing device is described as follows. 
The operations of the light emitting means 22, the light gathering means 23, the 

35 fingerprint acquiring means 24, the characteristic point extracting means 25, the memory 
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means 26 and the movement detecting means 27 are the same as described above. 
However, when a pointing function is performed, the operating means 31 receives 
displacement data on characteristic points of fingerprint images or fingerprint image sets 
calculated in the movement detecting means 27, and calculates a direction and a distance 
5 where the pointer is to move on a monitor with the displacement data. That is, as shown 
in Fig. 4, the operating means 31 calculates a desired direction and a desired distance 
where the pointer is to move. Here, although the movement detecting means 27 
calculates displacement data with characteristic points of the acquired fingerprint image, 
displacement data can be directly calculated with digital fingerprint image data. 
10 Fig. 8 is a flow chart illustrating a fingerprint recognition process in a pointing 

device according to the first or the second embodiment of the present invention. 

If n ix set to be 1 (S802) after the pointing device is initialized (S801), each of 
the fingerprint acquiring means 24, 24-1, 2, 3 acquires n (n=l)th fingerprint image of 20 x 
20 pixels (S803). Here, when the plurality of fingerprint acquiring means 24-1, 2, 3 are 
15 used as shown in Fig. 6, the pointing device can acquire a plurality of fingerprint images 
(fingerprint image set) at the same time. As a result, although the size of the fingerprint 
image acquired from the respective fingerprint acquiring means at every time is about 20 
x 20 pixels, the whole size of the fingerprint image acquired at every time can be adjusted 
by controlling the number of fingerprint acquiring means. 
20 The characteristic point extracting means 25 extracts at least one or more 

characteristic points from the n fingerprint image acquired by the fingerprint acquiring 
means 24, 24-1, 2, 3, and stores the characteristic points in the memory means 26 (S804). 

Next, the mapping means 28 maps the n fingerprint image in the virtual image 
space 29 with the extracted characteristic points (S805). 
25 The recognizing means 30 identifies whether the size of the whole fingerprint 

image mapped in the virtual image space 29 becomes a previously set size (S806). Here, 
the set size represents the minimum size required in user recognition. That is, although 
each fingerprint image acquired from the respective fingerprint acquiring means 24, 24-1, 
2, 3 has the size of about 20 x 20 pixels, the size of the fingerprint image is sufficient to 
30 obtain movement information of fingerprint images but insufficient to obtain information 
for user recognition. That is, the fingerprint image of 20 x 20 pixels is sufficient to 
obtain movement information of the fingerprint image, but a fingerprint image of-about " 
100 x 100 pixels is required for user recognition through the fingerprint image. As a 
result, the set size of the fingerprint image is about 100 x 100 pixels, which is the size of 
35 the virtual image space 29. 
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If the size of the whole fingerprint image mapped in the virtual image space 29 is 
smaller than the previously set size, that is, the size of the virtual image space 29, in the 
step S806, a variable n is increased by 1 (S807) and then fingerprint images are 
continuously obtained (S803 ~ S806). The fingerprint image acquiring process 
continues until the size of the whole fingerprint image mapped in the virtual image space 
29 reaches the previously set size. 

When the size of the whole fingerprint image mapped in the virtual image space 
29 reaches the previously set size in the step S806, the recognizing means 30 extracts at 
least one or more characteristic points from the whole fingerprint image mapped in the 
virtual image space 29 (S808). 

The recognizing means 30 compares characteristic points of the previously 
registered fingerprint image with those of the whole fingerprint image extracted in the 
step S808 (S809). 

The recognizing means 30 determines whether the characteristic points 
compared in the step S808 are identical or not (S810), and the recognizing means 30 
certifies a user if the characteristic points are identical (S811) but refuses user 
certification if not (S812). 

Fig. 9 is a detailed flow chart illustrating the process of mapping a fingerprint 
image in a virtual image space (S805) in Fig. 8. 

First, the first fingerprint image is mapped in a specific location of the virtual 
image space 29 (S901). Here, the first fingerprint image (or fingerprint image set) is 
preferably mapped at the center of the virtual image space 29. 

Then, when the second fingerprint image is obtained by the fingerprint acquiring 
means 24, 24-1, 2, 3 in the next cycle, the movement detecting means 27 receives the 
second fingerprint image (S902) to calculate displacement data (distance and direction) of 
the second fingerprint image from the first fingerprint image (S903). Here, the second 
fingerprint image is a fingerprint image obtained with a predetermined time interval 
depending on movement of the fingerprint. The displacement data of the step S903 are 
calculated with movement information of the characteristic points of the first fingerprint 
image and the second fingerprint image. Thereafter, the mapping means 28 maps the 
second fingerprint image in a corresponding location of the virtual image space 29 
depending on the displacement data calculated by the movement detecting means ' 27 *" 
(S904). 

The fingerprint acquisition, the displacement data calculation and the mapping 
operation are continuously performed n times until the size of the whole fingerprint image 



reaches the previously set size, that is, the size of the virtual image space 29 (S905 ~ 
S908). 

As shown in Figs. 8 and 9, fingerprint images of about 20 x 20 pixels acquired n 
times depending on the set cycle are synthesized into a large fingerprint image to have a 
size required in user recognition, for example, about 100 x 100 pixels. The image 
having the size required in user recognition can be obtained by synthesizing fingerprint 
images acquired in each location and their relative movement information. 

Fig. 10 is a flow chart illustrating a process of controlling a pointer in a pointing 
device according to the present invention. 

If the finger 20 touches on the transparent member 21 (SI 001), the n th fingerprint 
image is obtained by the fingerprint acquiring means 24, 24-1, 2, 3 (S1002). Then, the 
(n+l) th fingerprint image is obtained by the fingerprint acquiring means 24, 24-1, 2, 3 
depending on the previously set cycle (SI 003). The movement detecting means 27 
calculates the degree of movement from the n lh fingerprint image to the (n+l) th fingerprint 
image, that is, displacement data (SI 004). The operating means 31 operates coordinate 
values of the pointer with displacement data, that is, direction and distance of movement' 
(SI 005). Next, a processor (not shown) combined with the operating means 31 moves 
the pointer corresponding to the coordinates values of the pointer calculated by the 
operating means 31 (S1006). 

In this way, in Figs. 3 and 6, the pointing device maps a plurality of fingerprint 
images each having a size suitable for pointer control and acquired by the fingerprint 
acquiring means 24, 24-1,2,3, and extends to have a size suitable for user recognition. 
As a result, the user recognition and the pointer control can be simultaneously performed 
with one kind of fingerprint recognizing sensor. 

Fig. 1 1 is a diagram illustrating a structure of a pointing device according to a 
third embodiment of the present invention. 

The pointing device of Fig. 11 comprises a light emitting means 22, a light 
gathering means 23, a fingerprint acquiring means 34, a fingerprint recognizing unit 100 
and a pointing control unit 200. 

The fingerprint recognizing unit 100 comprises a characteristic point extracting 
means 35 and a recognizing means 36, and the pointing control unit 200 comprises a 
fingerprint image extracting means 37, a movement detecting means 38 and an operating" = 
means 39. Here, the pointing device according to an embodiment of the present 
invention may further comprises a housing (not shown) including the light emitting 
means 22, the light gathering means 23, the fingerprint acquiring means 34, the 



characteristic point extracting means 35, the movement detecting means 38 and the 
operating means 39, and comprising a transparent member 21 where a finger surface 
touches apart from the fingerprint acquiring means 34 at a predetermined distance. 
More preferably, the pointing device of Fig. 1 1 is suitably mounted in a portable terminal 
5 device. 

In the pointing device of Fig. 11, when the finger 20 touches the transparent 
member 21, the light emitting means 22 emits light to the surface of the finger 20. The 
light emitting means 22 includes at least one or more light emitting diodes (abbreviated as 
"LED"). 

1 0 The light gathering means 23 condenses light reflected from the surface of the 

finger 20 after the light is emitted from the light emitting means 22 to the finger 20. A 
common optical convex can be used as the light gathering means 23. 

The fingerprint acquiring means 34 acquires a fingerprint image of a finger 
surface for controlling a pointer with light condensed through the light gathering means 

15 23. The fingerprint acquiring means 34 converts the analog fingerprint image condensed 
by the light gathering means 23 into a digital fingerprint image to obtain a fingerprint 
image of M x N pixels. Here, the size of M x N pixels acquired by the fingerprint 
acquiring means 34 represents a size required in the user recognition. That is, the size of 
M x N pixels represents a size to perform a user recognition on the fingerprint image by 

20 using the fingerprint image acquired one time. The fingerprint acquiring means 34 
includes an optical sensor array where a plurality of CMOS image sensors (abbreviated as 
"CIS") are arranged two-dimensionally. Here, the fingerprint acquiring means 34 
acquires fingerprint images in the previously set cycle. The fingerprint acquiring means 
34 is manufactured to be suitable for a mini-portable device, and the CIS for acquiring a 

2 5 large fingerprint image of over about 100 x 100 pixels is used. In this way, as the 
fingerprint acquiring means 34 for user recognition, the CIS can be used which acquires a 
fingerprint image having various sizes generally ranging from 90 x 90 pixels to 400 x 400 
pixels. Accordingly, the size of the fingerprint image acquired by the fingerprint 
acquiring means 34 of the third embodiment of the present invention is different from that 

30 of the fingerprint image acquired by the fingerprint acquiring means 24 of the first or the 
second embodiment of the present invention. 

The light generated from the light emitting means 22 is mirrored on the surface " 
of the finger 20, and reflected depending on patterns of the finger 20 surface. The light 
reflected from the bottom surface of the finger forms a phase in the fingerprint acquiring 

35 means 34 through the light gathering means 23. The phase formed in the fingerprint 



acquiring means 34 is converted into a digital fingerprint image by the fingerprint 
acquiring means 34. Such fingerprint image acquisition is continuously performed at a 
rapid speed on a time axis. 

The fingerprint recognizing unit 100 extracts characteristic points from the 
fingerprint image acquired from the fingerprint acquiring means 34 in an operating cycle 
different from that of the fingerprint acquiring means 34, and performs a user recognition 
by comparing the extracted characteristic points with those of the previously registered 
fingerprint image. The fingerprint recognizing unit 100 generally compares the 
extracted characteristic points with those of the previously registered fingerprint image 
one to three times per second. That is, the fingerprint recognition process for user 
certification is performed by receiving 1-3 fingerprint images per second, extracting 
characteristic points from the received fingerprint images and comparing the extracted 
characteristic points with those of the previously registered fingerprint image. More 
preferably, the processing of the fingerprint recognition is performed at every second. 
The fingerprint recognition unit 100 comprises the characteristic point extracting means 
35 for extracting characteristic points from the acquired fingerprint image and the 
recognizing means 36 for performing the user recognition by comparing characteristic 
points of the previously registered fingerprint image with those extracted by the 
characteristic point extracting means 35. 

The pointing control unit 200 extracts a fingerprint image of m x n pixels (M, N 
> m, n) from the fingerprint image acquired from the fingerprint acquiring means 34 in an 
operating cycle different from that of the fingerprint recognizing unit 100 to detect 
movement information of the fingerprint image. The pointing control unit 200 
calculates displacement data with the detected movement information, and calculates a 
direction and a distance where the pointer is to move with the calculated displacement 
data. Preferably, the pointing control unit 200 detects movement information of 
characteristic points of the fingerprint image, and calculates displacement data of the 
characteristic points depending on the movement information. The pointing control unit 
200 calculates the movement direction and distance of the pointer corresponding to the 
displacement data of the characteristic points. 

The pointing control unit 200 extracts a fingerprint image of about 20 x 20 pixels 
from the fingerprint image acquired in the previous set cycle, calculates displacement data" 
of each fingerprint image and then calculates 2 -dimensional coordinates (AX, AY), that is, 
a 2-dimensional direction and distance where the pointer is to move with the displacement 
data. The pointing control unit 200 extracts fingerprint images 800-1200 times about 
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per second, and calculates displacement data of each fingerprint image extracted 
depending on the corresponding cycle. 

The fingerprint recognizing unit 100 and the pointing control unit 200 
individually operates depending on different operating cycles, respectively, to perform the 
5 user recognition and the pointer control operation. That is, while a user of the pointing 
device controls the pointer with the fingerprint image, the fingerprint recognizing unit 100 
performs a fingerprint certification on the user independently of the pointing control 
process. As a result, the fingerprint certification is periodically performed during 
navigation for the pointer control without an additional fingerprint recognizing process. 

10 Hereinafter, the operations of the fingerprint recognizing unit 100 and the 

pointing control unit 200 are described in detail. 

The characteristic point extracting means 35 extracts at least one or more 
characteristic points from the fingerprint images acquired at every time depending on the 
previously set cycle. These characteristic points includes a length, a direction of a 

15 fingerprint ridge and location data where the ridge is separated or ended. 

The recognizing means 36 compares characteristic points of the previously 
registered fingerprint image with those extracted from the characteristic point extracting 
means 35 to perform the user recognition depending on identification of the two 
fingerprint images. Here, the recognizing means 36 may include a comparing means 

20 (not shown) for combining global information and local characteristic information of the 
acquired fingerprint image and the previously registered fingerprint image or comparing 
the two fingerprint images with characteristic point matching on the two fingerprint 
images. The recognizing means 36 determines identification of the two fingerprint 
images with the comparing means. 

2 5 The recognizing means 36 performs a user recognition if characteristic points of 

the previously registered fingerprint image are identical with those extracted from the 
characteristic point extracting means 35, or refuses the user recognition if not. 

The fingerprint image extracting means 37 extracts a fingerprint image of m x n 
pixels (here, M, N > m, n) from the fingerprint image of M x N pixels acquired from the 

30 fingerprint acquiring means 34. The size of m x n pixels represents a size used in 
pointer control. The size of 20 x 20 pixels for the pointer control in the pointing device 
is sufficient for the fingerprint image to generally extract a small fingerprint image."- The" ' 
fingerprint image extracting means 37 extracts fingerprint images ranging from about 15 
x 15 pixels to about 80 x 80 pixels. As a result, the fingerprint image extracting means 

35 37 extracts a fingerprint image of about 20 x 20 pixels for the pointer control from the 
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fingerprint image of about 100 x 100 pixels acquired from the fingerprint acquiring 
means 34 for the user recognition. Here, the size of the used fingerprint image is just an 
example of the present invention. That is, the acquired size of M x N pixels is 
preferably suitable for the user recognition, and the extracted size of m x n pixels is 
5 preferably suitable for the pointer control. 

The movement detecting means 38 grasps the degree of movement of each 
fingerprint image acquired at every time depending on the set cycle. Here, the 
movement detecting means 38 preferably detects the degree of movement of the 
fingerprint with a motion estimation method by calculating displacement data (direction 

10 and distance) on characteristic points of the fingerprint images acquired in the set cycle. 
More preferably, the movement detecting means 38 detects the degree of movement of 
the fingerprint image by calculating displacement data on characteristic points of the 
fingerprint images acquired in the set cycle. Here, the displacement data of the 
fingerprint image are calculated by calculating movement distance and direction on 

1 5 characteristic points of the fingerprint image acquired in the current cycle from those of 
the fingerprint image acquired in the previous cycle. The movement information of the- 
fingerprint image and the characteristic point extraction in the fingerprint recognition as 
well as the fingerprint acquisition are importance factors because the movement of 
fingerprint images and the reliability of the fingerprint recognition are differentiated 

2 0 depending on how reliably characteristic points can be extracted. 

The operating means 39 receives movement degree of the fingerprint image from 
the movement detecting means 38, that is, displacement data, and calculates 2- 
dimensional coordinates (AX, AY), that is, direction and distance/movement degree where 
the pointer is to move with the received displacement data. 

2 5 The operating means 39 is generally combined with a pointing device or with a 

processor of apparatus having the pointing device. As a result, the processor can control 
the movement of the pointer on a screen of a display device depending on the coordinates 
calculated in the operating means 39. 

The pointing device according to an embodiment of the present invention may 
30 further comprise a display means (not shown) for displaying the previously stored 
information. In this way, when the pointing device further comprises the display means, 
the display means receives signals depending on the fingerprint recognition - of- the- ~ 
fingerprint recognizing unit 100 to display the recognition result. When the recognition 
on the user is successfully performed in the fingerprint recognizing unit 100, information 

3 5 for performing all functions of the corresponding terminals are displayed on the display 
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means. However, when the user recognition is refused, restrictive information is 

displayed which can perform only a specific function of the terminal. 

The technology of restrictively allowing usage of the corresponding terminal 

through the user recognition will be mentioned later. 
5 The fingerprint acquiring means 34 in the third embodiment of the present 

invention can be embodied with a semiconductor device as shown in the first and the 

second embodiments of the present invention. 

Meanwhile, when the fingerprint acquiring means 34 is embodied with an optical 

system, a large fingerprint image for user recognition can be obtained with a mini- 
10 fingerprint acquiring means, that is, 'reduced optical system'. In other words, the size of 

the fingerprint acquiring means 34 can be miniaturized by reducing the size of the actual 

fingerprint by 1/2-1/4 and acquiring the reduced fingerprint image. 

The principle and the process of acquiring a fingerprint image with the reduced 

optical system are described in detail. 
15 Fig. 12 is a diagram illustrating a design example of 3 : 1 reduction optics 

mountable in a microscopic space according to the present invention. As shown in Fig. 

12, aspherics 42 are used to mount the optical device in the microscopic space. If the 

optical system is configured as shown in Fig. 12, the size of the actual object represented 

by a left arrow 41 is reduced to about a 1/3 size represented by a right arrow 43. If an 
2 0 image of the object represented by the left arrow 41 passes through the aspherics 42, an 

inverse image is formed with about a 1/3 size in the fingerprint acquiring means 34 

located at the right side. The reduced optical system is embodied by application the 

above-described principle so that the size of the actual fingerprint is reduced to the size of 

1/n (here, n is a real number ranging from 1 to 5. 

2 5 Fig. 13 is a diagram illustrating an example of a fingerprint image acquired from 

a fingerprint acquiring means by applying the reduction optic system of Fig. 12. Fig. 
13a shows a fingerprint image acquired from the fingerprint acquiring means 34 at the 
optical system of 1 : 1, and Fig. 13b shows a fingerprint image acquired from the 
fingerprint acquiring means 34 at the reduced optical system of 4 : 1 . 
30 Generally, about 2 valleys are formed at every 1mm in a human fingerprint. As 

a result, a recognition pixel of the fingerprint acquiring means 34 is 0.5mm, and the 
number of the fingerprint acquired by the fingerprint acquiring means 34 is just-2V"as " 
shown in Fig. 13a, when the fingerprint acquiring means 34 of 20 x 20 pixels is used. In 
this way, as the acquired number of valleys of the fingerprint image becomes smaller, the 

3 5 accuracy is degraded in user recognition and the performance can be also degraded in 
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pointer control. As a result, the size of the sensor is required to be larger for sufficient 
data collection. 

In order to overcome the above-described problem, much more data can be 
obtained by applying the reduced optical system to the fingerprint acquiring means 34 
5 without enlarging the size of the sensor. That is, in the embodiment of the present 
invention, the size of the fingerprint having an interval 0.5mm is acquired by reducing the 
size of the fingerprint by 1/n (here, n is a real number ranging from 1 to 5). More 
specifically, the size of the fingerprint is reduced to the size of 1/2 ~ 1/4. As a result, 
much more data can be obtained with the fingerprint acquiring means 34 having the same 

10 size of that in Fig. 13a than in Fig. 13a. As shown in Fig. 13b, when the size of the 
fingerprint is reduced to the size of 1/4 with the reduced optical system, the fingerprint 
interval of 0.5mm can be reduced to about 0.125mm. Therefore, 4-16 times fingerprint 
information can be obtained with the fingerprint acquiring means 34 having the same size 
in comparison with Fig. 13a. In other words, when a fingerprint image is obtained by 

15 reducing a fingerprint having an average interval of 0.5mm to a 1/2-1/4 size , the size of 
the fingerprint acquiring means 34 can be miniaturized to 1/4-1/16. As a result, it is 
possible to obtain a fingerprint image for user recognition and pointer control with the 
miniaturized fingerprint acquiring means 34 of low power consumption and low cost, and 
the miniaturized fingerprint acquiring means 34 is advantageous in application to a mini- 

2 0 portable terminal device. 

Fig. 14 is a diagram illustrating a pointing device having a fingerprint 
recognizing function according to a fourth embodiment of the present invention. 

The pointing device of Fig. 14 further comprises a storing means 60 in 
comparison with that of Fig. 11. The storing means 60 stores fingerprint images 

2 5 acquired from the fingerprint acquiring means 34. In the pointing device of Fig. 14, if 
the fingerprint images acquired from the fingerprint acquiring means 34 at every time 
depending on the previously set cycle are stored in the storing means 60, the fingerprint 
recognizing unit 100 and the pointing control unit 200 individually perform a user 
recognition and a pointer control with the fingerprint images stored in the storing means 

30 60. That is, while the user recognition and the pointer control are performed in the 
fingerprint recognizing unit 100 and the pointing control unit 200, respectively, which 
immediately receive the fingerprint images acquired depending on the operating cycle'of 
the fingerprint acquiring means 34 in the pointing device of Fig. 11, the fingerprint 
images acquired from the fingerprint acquiring means 34 are first stored in the storing 

35 means 60 and then the pointer control is performed only with the fingerprint image 
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having a size required in the pointer control so that the pointer may be embodied with low 
cost, low power consumption and high-speed navigation information production. Here, 
it is natural to simultaneously perform the user recognition and the pointer control with 
the fingerprint images stored in the storing means 60. 
5 In this way, the user recognition and the pointer control can be simultaneously 

performed with one kind of fingerprint recognizing sensors by extracting a fingerprint 
image having a size required in the pointer control from the fingerprint images acquired 
from the fingerprint acquiring means 34 for the user recognition in the pointing devices of 
Figs. 11 and 14. 

10 Fig. 15 is a diagram illustrating a method for extracting a fingerprint image of m 

x n pixels from that of M x N pixels. 

The fingerprint acquiring means 34 acquires a fingerprint image 71 ofMxN 
pixels depending on a predetermined cycle. The fingerprint image 71 of M x N pixels 
has a sufficient size for user recognition. Preferably, the fingerprint image 71 has a size 

15 ranging from 90 x 90 pixels to 400 x 400 pixels. Also, the fingerprint image extracting 
means 37 extracts a fingerprint image 72 of m x n pixels from the fingerprint image 71 of 
M x N pixels. Here, the fingerprint image extracting means 37 extracts a central portion 
of the fingerprint image 71 of M x N pixels. The size of m x n pixels represents the size 
of the fingerprint image 72 where the pointer control is possible. The fingerprint image 

20 72 has a size ranging from 15x15 pixels to 80 x 80 pixels. 

In the pointing device of Fig. 14, the fingerprint image 71 of M x N pixels is 
stored in the storing means 60, and the user recognition is performed with the fingerprint 
image 71 of M x N pixels. At the same time, the pointer control is performed with the 
fingerprint image 72 of m x n pixels extracted from the fingerprint image 71 ofMxN 

25 pixels. 

In case that the pointer is regulated with the fingerprint image 72 of m x n pixels, 
when the surface of the finger 20 moves from a first location to a second location by AX 
and A Y, data on the fingerprint image of m x n pixels extracted from the fingerprint 
image extracting means 37 are transmitted to the movement detecting means 38. Here, 

30 the fingerprint image is transmitted at a speed of about 800-1200 times per second. As 
a result, displacement data of the fingerprint image depending on movement of the finger 
20 are calculated and converted to the speed, and the movement direction and distance' of " 
the pointer are also calculated and converted to the speed. Here, since the above- 
described processing speed is required for a stable pointing operation in the pointing 

35 device according to an embodiment of the present invention, it is preferable to select the 
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minimum image size in order to reduce the processing and calculation amount. 

Meanwhile, the whole fingerprint image 71 of M x N pixels required for the user 
recognition is transmitted to the fingerprint recognizing unit 100. The fingerprint image 
71 is transmitted at a speed of 1-3 times per second where a general recognition 
5 processing can be performed. The fingerprint recognizing unit 100 is configured to be 
included in a processing device of the portable terminal device so that the fingerprint 
recognizing unit 100 can perform the function. 

The process of calculating the displacement data of the fingerprint image 72 with 
the characteristic points of the fingerprint image 72 is the same as that of Fig. 4. 
10 Fig. 16 is a flow chart illustrating a method for performing a user recognition and 

a pointing control at the same time according to the third or the fourth embodiment of the 
present invention. 

At the initialization state (S1601), if the surface of the finger 20 touches the 
transparent member 21 (SI 602), the fingerprint image of M x N pixels is obtained by the 
1 5 fingerprint acquiring means 34 depending a first operating cycle (SI 603). 

The fingerprint recognizing unit 100 and the pointing control unit 200 
simultaneously perform the user recognition (S1620) and the pointer control (S1630) with 
the fingerprint image 71 obtained by the fingerprint acquiring means 34. In case of the 
pointing device of Fig. 14, the user recognition (S1620) and the pointer control (S1630) 
2 0 are the same as those of the pointing device of Fig. 1 1 except in that the obtained 
fingerprint image 71 is stored in the storing means 60 and the fingerprint image 61 stored 
in the storing means 60 is used. 

In the user recognition process (SI 620), the characteristic point extracting means 
35 extracts at least one or more characteristic points from the fingerprint image of M x N 

2 5 pixels depending on a second operating cycle to transmit the characteristic points to the 

recognizing means 36 (SI 604). The recognizing means 36 compares the characteristic 
points of the previously registered fingerprint image with those extracted from the 
fingerprint image of M x N pixels (S1605). The recognizing means 36 determines 
whether the characteristic points of the two fingerprint images are identical from the 
30 comparison result (SI 606). The recognizing means 36 certifies a user (S1607) if the 
characteristic points of the two fingerprint images are identical, and refuses the user 
recognition (Si 608) if not. 

Next, in the pointer control process (SI 630), the fingerprint image extracting 
means 37 extracts the fingerprint image 72 of m x n pixels from the fingerprint image of 

3 5 M x N pixels depending on a third cycle to transmit the extracted fingerprint image to the 



movement detecting means 38 (S1609). In the embodiment, m and n ranges from 15 to 
80 to have a size suitable for the pointer control of the extracted fingerprint image 72. 
The movement detecting means 38 calculates displacement data of the fingerprint images 
72 of m x n pixels to transmit the displacement data to the operating means 39 (SI 6 10). 
Here, the movement detecting means 38 calculates displacement data by calculating 
movement degree, that is, distance and direction, of the fingerprint image acquired in the 
current cycle from that acquired in the previously cycle. Preferably, the displacement 
data depending on the movement degree of characteristic points of the extracted 
fingerprint images 72 are calculated. The operating means 39 operates coordinates 
where the pointer is to move with the displacement data calculated in the movement 
extracting means 38 (S 161 1). A processor (not shown) of the terminal device moves the 
pointer corresponding to the coordinates of the pointer (S 1612). 

As described in Fig. 16, the pointing device according to the embodiment of the 
present invention can simultaneously perform the user recognition and the pointer control 
by using the fingerprint image 71 acquired from one fingerprint acquiring sensor 34. 
The user recognizing process (SI 620) and the pointer control process (SI 630) are 
performed on different operating cycles which are previously set, and the two process 
SI 620 and S1630 are individually perform. That is, while a user regulates the pointer 
with the fingerprint (SI 630), the user recognition process (SI 620) is naturally performed. 
As a result, the fingerprint recognizing process is not required to a user, and the 
fingerprint is automatically recognized during the pointer control so that the range of 
service available by users can be regulated to the corresponding device depending on the 
fingerprint recognizing result. 

Fig. 17 is a diagram illustrating a structure of a pointing device according to a 
fifth embodiment of the present invention. 

In comparison with the above-described embodiments of the present invention, 
the fifth embodiment is characterized in that the process is not comprised to extract 
characteristic points from the fingerprint image during the fingerprint recognizing 
function. That is, a location where the acquired fingerprint image is stored in the storing 
means depending on the extracted characteristic points is not determined. Instead, a 
mapping location of the fingerprint image is determined depending on movement distance, 
that is, displacement data, and stored in the storing means. 

Hereinafter, the configuration of the pointing device of Fig. 17 is described in 

detail. 

The pointing device of Fig. 17 comprises a transparent member 21, a light 



emitting means 22, a light gathering means 23 and a fingerprint acquiring means 34. 
However, the explanation on the configuration of these elements is omitted because it is 
the same as that of the third or the fourth embodiment. 

The fingerprint image obtained by the fingerprint acquiring means 34 is 
5 immediately input into a pointing control unit 200 including a fingerprint image 
extracting means 37, a movement detecting means 38 and an operating means 39. Since 
the operation of the detailed elements of the pointing control unit 200 is the same as that 
described in the third or the fourth embodiment, the specific explanation is omitted. 

However, in comparison with the third or the fourth embodiment, the operation 

10 of the fifth embodiment is characterized in that the fingerprint image of m x n pixels 
extracted by the fingerprint image extracting means 37 is stored in the storing means 40 
and the storing location is mapped depending on data of displacement values calculated 
by the operating means 39. Specifically, suppose that the i th fingerprint image extracted 
from the fingerprint image extracting means 37 is stored in a specific location of the 

15 storing means 40. If the (i+l) th fingerprint image is extracted from the fingerprint image 
extracting means 37, the displacement data AX and AY obtained through the movement 
detecting means 38 and the operating means 39 are received, and the (i+l) th fingerprint 
data are stored in a location moved by the displacement data from the specific location 
where the i th fingerprint data are stored. 

2 0 The operation of storing the fingerprint data in the storing means 40 is performed 

by a method of periodically storing data depending on a predetermined time interval or 
performed when a specific command is received. 

Also the fifth embodiment of the present invention includes a CPU 50 for 
controlling the operation of storing fingerprint data in the above-described storing means 

2 5 40, controlling the movement of the pointing device by receiving the displacement data 

AX and AY from the operating means 39 and performing the fingerprint recognizing 
operation described later in Fig. 18. 

Fig. 18 is a flow chart illustrating the operation of the pointing device according 
to the fifth embodiment of the present invention. 

3 0 If the system is initialized and the surface of the finger 20 touches the transparent 

member 21 (SI 8 10), the fingerprint image of m x n pixels is acquired by the fingerprint 
acquiring means 34 and the fingerprint image extracting means 37 (SI 820). 

The acquired fingerprint image is stored in the storing means 40 (SI 830), and the 
displacement data and the pointer coordinates on the fingerprint image are calculated 
3 5 (SI 840, SI 850). The calculation result is provided to the storing means 40, and used in 



mapping the fingerprint image. 

The calculation result is also provided to the CPU 50, and used to control the 
operation of the pointing device (SI 860, SI 870). 

Meanwhile, the CPU 50 receives the fingerprint image from the storing means 
40. If the received fingerprint image is not a fingerprint image of M x N pixels, the 
CPU 50 receives a fingerprint image from the storing means 40 again (S1880). If the 
received fingerprint image is identical with that of M x N pixels, the fingerprint image is 
compared with the previously stored fingerprint image (SI 890), and identification of the 
two fingerprint images is determined (SI 899). Then, the user recognition or recognition 
refusal operations are performed depending on the result. 

Fig. 19 is a flow chart illustrating a method for limiting usage of a portable 
communication terminal device depending on users by using a fingerprint recognition 
function according to the present invention. 

In an embodiment, the portable terminal device having a fingerprint recognition 
function identifies fingerprints of users to perform a fingerprint recognition on the users 
(SI 900). The portable terminal device identifies through the fingerprint recognition 
whether a person who intends to user the terminal device is the person himself or herself 
(S1910). When a user is the person himself or herself, the terminal device displays the 
whole menu that can be provided by the terminal device (SI 920) so that the user may use 
all functions (service) (SI 930). However, when the user is not the person himself or 
herself, the terminal device only displays a specific menu that can allowed to be used by 
the person himself or herself (SI 940). In this way, important information such as credit 
and finance information can be protected by limiting usage of the terminal device to a 
person who is not recognized through the fingerprint recognition. 

If a user who is not the person himself or herself intends to use functions which 
are not allowed (SI 950), the terminal device displays a message that the corresponding 
function cannot be used (SI 960). Then, after a few seconds, the terminal device 
displays again a menu whose usage is allowed by the person himself or herself (SI 940). 
Accordingly, in case that a person is not a previously registered user, personal 
information of the user, control information of the system and pay service such as e- 
commerce can be protected by limiting access of other users for personal information 
protection. 

The fingerprint identifying process can be operated as a background process in 
order to relieve inconvenience in usage of the portable communication terminal device. 
Here, the background process is to automatically perform steps of collection, analysis and 



identification of data required in fingerprint identification while a user uses a 2- 
dimensional pointer with a finger of the user without notifying the steps to the user. 
When the user uses a process previously specified for protection in the system or when an 
approval transaction is performed, a required process is performed in case of the person 
himself or herself with data obtained from the background process. However, when the 
user is not the person himself or herself, the process is refused to protect information or 
possession of a possessor. 

That is, the person identification process is performed in combination with the 
background process. The next step is successively performed when the user is the 
person himself or herself, but the subsequent usage is refused when the user is not the 
person himself or herself. As a result, the necessary stability can be secured without 
affecting convenience of users. Here, the same sensor of package in fingerprint 
registration and identification as that of the 2-dimensional pointing device is used, and the 
pointing device is configured to perform the data collection and the software process at 
the same time. 

In the above-described embodiment, if the person himself or herself registers his 
or her fingerprint, the registered data are saved in a non-volatile memory and repeatedly 
used until the person himself or herself changes the data. 

In another embodiment of the present invention, not a fingerprint recognition 
sensor for fingerprint recognition but a pointing device having a fingerprint certifying 
sensor function can identify through fingerprint identification of users whether a portable 
terminal device user is the person himself or herself while the user uses the portable 
communication terminal device. For this operation, the portable terminal device collects 
2-dimensional movement information while user moves his or her finger, generates a 2- 
dimensional image having a size required for the person identification by synthesizing the 
collected movement information and the fingerprint image in the corresponding location, 
and extracts characteristic points from the corresponding fingerprint image. Then, the 
portable terminal device registers the extracted characteristic points or compares the 
extracted characteristic points with registered data for the person identification. 

Fig. 20 is a diagram illustrating an example of a portable terminal device 
comprising a pointing device according to the present invention. The portable terminal 
device includes a cellular phone, a PDA or a smart phone. 

In the portable terminal device, an external surface of a transparent member 230 
is exposed, and a fingerprint image having a size required for user recognition is obtained 
through fingerprint acquisition when a finger is put on the external surface of the 
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transparent member 230. If the fingerprint image is acquired, the existing menu window 
is changed to a service screen 240 as shown in Fig. 20. 

In the portable terminal service, the service can be used by selecting and clicking 
the menu on the service screen 240 not with a moving key such as a mouse of a general 
5 computer but with a pointer 250. Also, the portable terminal device comprises at least 
one or more function buttons 220 for performing other functions or inputting performance 
commands. 

While the invention is susceptible to various modifications and alternative forms, 
specific embodiments have been shown by way of example in the drawings and described 
10 in detail herein. However, it should be understood that the invention is not limited to the 
particular forms disclosed. Rather, the invention covers all modifications, equivalents, 
and alternatives falling within the spirit and scope of the invention as defined in the 
appended claims. 

1 5 INDUSTRIAL APPLICABILITY 

In a pointing device having a fingerprint recognition function and a fingerprint 
recognition method according to an embodiment of the present invention, fingerprint 
images having small sizes are mapped to generate a large fingerprint image or a small 
fingerprint image extracted from the large fingerprint image so that the pointing device 

20 performs user recognition and pointer control. As a result, respective sensors for the 
user recognition and for the pointer control are not comprised in the pointing device, but 
only one kind of sensors for performing both functions of user recognition and pointer 
control is comprised in the pointing device according to an embodiment of the present 
invention. Also, it is possible to easily embody miniaturization of a portable terminal 

2 5 device because the user recognition which requires a large fingerprint can be performed 
only with a small fingerprint image, thereby reducing manufacturing cost. 

Additionally, important information in the terminal device can be protected by 
limiting kinds of service usable in the terminal device depending on user recognition. 
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